Computer-readable storage medium, information processing apparatus, information processing system, and information processing method

ABSTRACT

An exemplary game system includes a monitor for displaying a game image or a television program, and a terminal device having a camera, a microphone, and a loudspeaker. When a user is playing a game or viewing a television program by using the monitor, another user can have chat by using the terminal device.

CROSS REFERENCE TO RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2011-125647, filed onJun. 3, 2011, is incorporated herein by reference.

FIELD

The exemplary embodiments relate to a computer-readable storage media,information processing apparatuses, information processing systems, andinformation processing methods, and particularly relate tocomputer-readable storage media, information processing apparatuses,information processing systems, and information processing methods thatenable chat between users at distant locations.

BACKGROUND AND SUMMARY

Conventionally, there is known a game system that allows a user playinga network game to chat with another user playing the network gametogether with the user.

Further, there is conventionally known a communication system in which acommunication space is provided in a message exchange device based onattribute information about a broadcast program sent from a broadcaststation, messages are exchanged between terminal devices by means of thecommunication space, and an image of the broadcast program andinformation about the message exchange are displayed on screens of imagedisplay devices connected to the terminal devices.

In the above game system, however, not only a game image, but also aninput box for inputting a character string desired to be transmitted tothe other user, and a chat box for displaying a character stringreceived from the other user, are displayed on a screen of a monitor.This makes it difficult to view the game image.

Also in the above communication system, since the image of the broadcastprogram and the information about the message exchange are displayed onthe screens of the image display devices, it becomes difficult to viewthe broadcast program.

An object of the exemplary embodiments is to provide a computer-readablestorage medium, an information processing apparatus, an informationprocessing system, and an information processing method that allow,without affecting playing of a game or viewing of a television program,users at distant locations to communicate in real time with each otherwhile viewing images displayed on screens as in the case of video chatand text chat.

The above object is attained by, for example, the followingconfigurations.

A first aspect is a computer-readable storage medium having storedtherein an information processing program executed in an informationprocessing apparatus that is capable of displaying images on astationary display device and on a portable display device having apredetermined input section, and that includes an input device and acommunicator configured to communicate with another informationprocessing apparatus (hereinafter, referred to as communicationcounterpart apparatus). The information processing program causes acomputer of the information processing apparatus to function as aprocessor, a first display controller, a transmitter, a receiver, and asecond display controller.

The processor executes a predetermined information process, based on aninput from the input device. The first display controller causes animage which is based on the predetermined information process to bedisplayed on the stationary display device. The transmitter transmits,to the communication counterpart apparatus through the communicator,character information or a camera image inputted via the predeterminedinput section. The receiver receives, from the communication counterpartapparatus through the communicator, character information or a cameraimage inputted on the communication counterpart apparatus. The seconddisplay controller causes the character information or the camera imagethat is received by the receiver to be displayed on the portable displaydevice.

The predetermined information process may be a game process.

Further, the transmitter may transmit game data for a network gameprocess to the communication counterpart apparatus through thecommunicator, in addition to the character information or the cameraimage that is inputted via the predetermined input section. The receivermay receive game data for the network game process from thecommunication counterpart apparatus through the communicator, inaddition to the character information or the camera image that isinputted on the communication counterpart apparatus. The processor mayexecute the network game process, based on the input from the inputdevice and the game data that is received from the communicationcounterpart apparatus.

Further, the information processing apparatus may be connected to afirst loudspeaker, the predetermined input section may include a cameraand a microphone, and the portable display device may include a secondloudspeaker. In addition, the transmitter may transmit, to thecommunication counterpart apparatus through the communicator, a cameraimage inputted via the camera and a microphone sound inputted via themicrophone. The receiver may receive, from the communication counterpartapparatus through the communicator, a camera image and a microphonesound that are inputted on the communication counterpart apparatus. Theinformation processing program may further cause the computer tofunction as a first sound controller and a second display controller.The first sound controller generates a game sound in accordance with thegame process, and causes the game sound to be outputted from the firstloudspeaker. The second sound controller causes the microphone soundthat is received by the receiver to be outputted from the secondloudspeaker.

Further, the second display controller may include: an image generatorconfigured to generate an image to be displayed on the portable displaydevice, based on the character information or the camera image that isreceived by the receiver; and an image outputter configured to outputthe image generated by the image generator to the portable displaydevice by wireless transmission.

Further, the second display controller may include: an image generatorconfigured to generate an image to be displayed on the portable displaydevice, based on the character information or the camera image that isreceived by the receiver; and an image outputter configured to compressthe image generated by the image generator, and to output the compressedimage to the portable display device.

Further, the predetermined input section may include a touch panel, andthe character information may be inputted by means of the touch panel.

Further, the stationary display device may have a function of receivingand displaying television broadcasts. In addition, the portable displaydevice may have a function of outputting a control signal to thestationary display device in accordance with a channel selectioninstruction inputted through the predetermined input section, and thusswitching television channels to be displayed on the stationary displaydevice.

Further, the information processing apparatus may be communicable with aserver device through the communicator. In addition, the informationprocessing program may further cause the computer to function as anuploader, a downloader, and a communication counterpart determiner. Theuploader uploads, into the server device, channel information indicatinga television channel that a user of the information processing apparatusis viewing, and user identification information for identifying theuser, in accordance with the channel selection instruction. Thedownloader downloads, from the server device, user identificationinformation that is associated with the channel information uploaded bythe uploader among user identification information uploaded from otherinformation processing apparatuses into the server device. Thecommunication counterpart determiner determines the communicationcounterpart apparatus, based on the user identification informationdownloaded by the downloader.

Further, the information processing apparatus may further include astorage section configured to store user identification informationabout users of other information processing apparatuses. In addition,the information processing program may further cause the computer tofunction as a communication counterpart determiner configured todetermine the communication counterpart apparatus, based on the useridentification information stored in the storage section about the usersof the other information processing apparatuses.

Further, the communication counterpart determiner may select useridentification information among the user identification informationstored in the storage section about the users of the other informationprocessing apparatuses, based on an input from the input device or thepredetermined input section, and may determine, as the communicationcounterpart apparatus, another information apparatus that is associatedwith the selected user identification information.

Further, the predetermined input section may include a microphone, andthe portable display device may include a loudspeaker. The transmittermay transmit a microphone sound inputted via the microphone, to thecommunication counterpart apparatus through the communicator, inaddition to the character information or the camera image. The receivermay receive a microphone sound inputted on the communication counterpartapparatus, from the communication counterpart apparatus through thecommunicator, in addition to the character information or the cameraimage that is inputted on the communication counterpart apparatus. Theinformation processing program may further cause the computer tofunction as a sound controller configured to cause the microphone soundthat is received by the receiver to be outputted from the loudspeaker.

A second aspect is an information processing apparatus that is capableof displaying images on a stationary display device and on a portabledisplay device having a predetermined input section. The informationprocessing apparatus includes: an input device; a communicatorconfigured to communicate with another information processing apparatus(hereinafter, referred to as communication counterpart apparatus); aprocessor configured to execute a predetermined information process,based on an input from the input device; a first display controllerconfigured to cause an image which is based on the predeterminedinformation process to be displayed on the stationary display device; atransmitter configured to transmit, to the communication counterpartapparatus through the communicator, character information or a cameraimage inputted via the predetermined input section; a receiverconfigured to receive, from the communication counterpart apparatusthrough the communicator, character information or a camera imageinputted on the communication counterpart apparatus; and a seconddisplay controller configured to cause the character information or thecamera image that is received by the receiver to be displayed on theportable display device.

A third aspect is an information processing system that is capable ofdisplaying images on a stationary display device and on a portabledisplay device having a predetermined input section. The informationprocessing system includes: an input device; a communicator configuredto communicate with another information processing system (hereinafter,referred to as communication counterpart system); a processor configuredto execute a predetermined information process, based on an input fromthe input device; a first display controller configured to cause animage which is based on the predetermined information process to bedisplayed on the stationary display device; a transmitter configured totransmit, to the communication counterpart system through thecommunicator, character information or a camera image inputted via thepredetermined input section; a receiver configured to receive, from thecommunication counterpart system through the communicator, characterinformation or a camera image inputted on the communication counterpartsystem; and a second display controller configured to cause thecharacter information or the camera image that is received by thereceiver to be displayed on the portable display device.

A fourth aspect is an information processing method performed by aninformation processing apparatus that is capable of displaying images ona stationary display device and on a portable display device having apredetermined input section, and that includes an input device and acommunicator configured to communicate with another informationprocessing apparatus (hereinafter, referred to as communicationcounterpart apparatus). The information processing method comprises thesteps of: executing a predetermined information process, based on aninput from the input device; causing an image which is based on thepredetermined information process to be displayed on the stationarydisplay device; transmitting, to the communication counterpart apparatusthrough the communicator, character information or a camera imageinputted via the predetermined input section; receiving, from thecommunication counterpart apparatus through the communicator, characterinformation or a camera image inputted on the communication counterpartapparatus; and causing the character information or the camera imagethat is received from the communication counterpart apparatus to bedisplayed on the portable display device.

The exemplary embodiments described herein allow, without affectingplaying of a game or viewing of a television program, users at distantlocations to communicate in real time with each other while viewingimages displayed on screens as in the case of video chat and text chat.

These and other objects, features, aspects and advantages of theexemplary embodiments will become more apparent from the followingdetailed description when taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an external view showing a non-limiting example of a gamesystem 1 according to an embodiment;

FIG. 2 is a functional block diagram showing a non-limiting example of agame apparatus body 5 shown in FIG. 1;

FIG. 3 shows a non-limiting example of an external structure of aterminal device 6 shown in FIG. 1;

FIG. 4 shows a non-limiting example of a state in which a user holds theterminal device 6;

FIG. 5 is a block diagram showing a non-limiting example of an internalstructure of the terminal device 6 shown in FIG. 3;

FIG. 6 shows a non-limiting example of a state in which a plurality ofgame apparatus bodies are connected to each other through a network;

FIG. 7 shows non-limiting examples of images that are displayed on amonitor 2 and an LCD 61 when a network game is being played;

FIG. 8 shows a non-limiting example of a terminal game image displayedon the LCD 61;

FIG. 9 shows a non-limiting example of a video chat image displayed onthe LCD 61;

FIG. 10 shows non-limiting examples of images that are displayed on themonitor 2 and the LCD 61 when a network game is being played and videochat is being conducted;

FIG. 11 shows non-limiting examples of images that are displayed on themonitor 2 and the LCD 61 when a television program is being viewed;

FIG. 12 shows a non-limiting example of a monitor control imagedisplayed on the LCD 61;

FIG. 13 shows a non-limiting example of a chat image displayed on theLCD 61;

FIG. 14 shows a non-limiting example of a chat image displayed on theLCD 61;

FIG. 15 shows a non-limiting example of a chat image displayed on theLCD 61;

FIG. 16 shows a non-limiting example of a memory map of an external mainmemory 12;

FIG. 17 shows a non-limiting example of a flowchart illustrating theflow of information processing based on information processing programD1;

FIG. 18 shows a non-limiting example of a flowchart illustrating theflow of a network game process; and

FIG. 19 shows a non-limiting example of a flowchart illustrating theflow of a monitor control process.

DETAILED DESCRIPTION OF NON-LIMITING EXAMPLE EMBODIMENTS FirstEmbodiment

A game system according to an embodiment (first embodiment) will bedescribed with reference to FIG. 1.

In FIG. 1, a game system 1 includes a household television receiver(hereinafter, referred to as a monitor) 2 which is an example of displaymeans, and a stationary game apparatus 3 connected to the monitor 2 viaa connection cord. The game apparatus 3 includes an optical disc 4, agame apparatus body 5, a terminal device 6, and a controller 7.

The optical disc 4 has stored therein an information processing program(typically, a game program) to be executed by the game apparatus body 5.

The monitor 2 displays a game image outputted from the game apparatusbody 5. The monitor 2 includes loudspeakers 2R and 2L. The loudspeakers2R and 2L each output a game sound outputted from the game apparatusbody 5.

The game apparatus body 5 executes a game process and the like based on,for example, the game program stored in the optical disc 4.

The controller 7 includes a plurality of operation sections (operationbuttons). The controller 7 transmits, to the game apparatus body 5,operation data (controller operation data) representing input states ofthe operation sections (whether or not each of the operation buttons hasbeen pressed), by using, for example, the Bluetooth (registeredtrademark) technology.

The controller 7 includes an imaging section for taking an image of amarker 8 consisting of two LED modules (hereinafter, referred to as“markers”) 8L and 8R provided in the vicinity of the display screen ofthe monitor 2 (above the screen in FIG. 1), and an imaging informationcalculation section for calculating the position of the marker in animage taken by the imaging section. The position of the markercalculated by the imaging information calculation section is transmittedas marker coordinate data to the game apparatus body 5. The gameapparatus body 5 can calculate the movement, position, attitude and thelike of the controller 7, based on the marker coordinate data.

In addition, the controller 7 includes an acceleration sensor and a gyrosensor. Acceleration data representing accelerations detected by theacceleration sensor, and angular velocity data representing angularvelocities detected by the gyro sensor, are transmitted to the gameapparatus body 5. The game apparatus body 5 can calculate the directionand movement of the controller 7, based on the acceleration data and/orthe angular velocity data.

The terminal device 6 is a portable device that is small enough to beheld by a user, and the user is allowed to move the terminal device 6with hands, or place the terminal device 6 at any location. Although aspecific structure of the terminal device 6 will be described later, theterminal device 6 includes an LCD (Liquid Crystal Display) 61 as displaymeans, and input means (a touch panel 62, a gyro sensor 604, and thelike described later). The terminal device 6 and the game apparatus body5 (a terminal communication module 28 (see FIG. 2)) are communicablewith each other wirelessly or via a cable. The terminal device 6receives, from the game apparatus body 5, data of an image (e.g., a gameimage) generated in the game apparatus body 5, and displays the imagerepresented by the data on an LCD 61. Although in the exemplaryembodiment an LCD is used as a display device, the terminal device 6 mayinclude any other display device, such as a display device utilizing EL(Electro Luminescence), for example. Further, the terminal device 6transmits, to the game apparatus body 5, operation data representing thecontent of an operation performed on the terminal device 6.

Next, with reference to FIG. 2, an internal structure of the gameapparatus body 5 will be described. FIG. 2 is a block diagramillustrating an example of an internal structure of the game apparatusbody 5. The game apparatus body 5 includes a CPU (Central ProcessingUnit) 10, a system LSI (Large Scale Integration) 11, an external mainmemory 12, a ROM/RTC (Read Only Memory/Real Time Clock) 13, a disc drive14, an AV-IC (Audio Video-Integrated Circuit) 15 and the like.

The CPU 10 is connected to the system LSI 11. In addition to the CPU 10,the external main memory 12, the ROM/RTC 13, the disc drive 14, and theAV-IC 15 are connected to the system LSI 11. The external main memory12, which is a volatile memory, is used as a work area and a buffer areafor the CPU 10. The ROM/RTC 13 includes a ROM (so-called boot ROM)incorporating a program for booting the game apparatus body 5, and aclock circuit (RTC) for counting time. The disc drive 14 reads, from theoptical disc 4, program data, texture data and the like, and writes theread data into an internal main memory 35 described below or theexternal main memory 12.

The system LSI 11 includes an input/output processor (I/O processor) 31,a GPU (Graphics Processor Unit) 32, a DSP (Digital Signal Processor) 33,a VRAM (Video RAM) 34, and the internal main memory 35.

The GPU 32 generates an image in accordance with a graphics command(draw command) issued by the CPU 10. In the exemplary embodiment, thegame apparatus body 5 may generate both a game image to be displayed onthe monitor 2 and a game image to be displayed on the terminal device 6.Hereinafter, the game image to be displayed on the monitor 2 may bereferred to as a “monitor game image”, and the game image to bedisplayed on the terminal device 6 may be referred to as a “terminalgame image”.

The DSP 33, serving as an audio processor, generates sound data by usingsound data and sound waveform (tone quality) data stored in the internalmain memory 35 and the external main memory 12. In the exemplaryembodiment, similarly to the game images, both a game sound to beoutputted from the loudspeakers 2R and 2L of the monitor 2 and a gamesound to be outputted from the loudspeakers of the terminal device 6 maybe generated. Hereinafter, the game sound to be outputted from themonitor 2 may be referred to as a “monitor game sound”, and the gamesound to be outputted from the terminal device 6 may be referred to as a“terminal game sound”.

Among the image data and sound data generated by the game apparatus body5 as described above, the image data and sound data to be outputted tothe monitor 2 are read by the AV-IC 15. Through an AV connector 16, theAV-IC 15 outputs the read image data to the monitor 2, and outputs theread sound data to the loudspeakers 2R and 2L included in the monitor 2.Thereby, an image is displayed on the monitor 2, and a sound isoutputted from the loudspeakers 2R and 2L.

Further, among the image data and sound data generated by the gameapparatus body 5, the image data and sound data to be outputted to theterminal device 6 are transmitted to the terminal device 6 by the I/Oprocessor 31 or the like. Data transmission to the terminal device 6 bythe I/O processor 31 or the like will be described later.

The I/O processor 31 executes data reception and transmission with thecomponents connected thereto, and download of data from an externalapparatus. The I/O processor 31 is connected to the flash memory 17, thenetwork communication module 18, the controller communication module 19,an extension connector 20, a memory card connector 21, and a codec LSI27. The codec LSI 27 is connected to the terminal communication module28.

The game apparatus body 5 is connected to a network such as the Internetso as to communicate with external information processing apparatuses(for example, other game apparatuses or various servers). That is, theI/O processor 31 is connected to a network via the network communicationmodule 18 and the antenna 22 so as to communicate with externalinformation processing apparatuses connected to the network. The flashmemory 17 may store not only the data transmitted and received betweenthe game apparatus body 5 and the external information processingapparatuses, but also saved data (result data or progress data of theprocess) of the game played with the game apparatus body 5. Further, theflash memory 17 may store programs such as a game program.

The game apparatus body 5 can receive operation data from the controller7. That is, the I/O processor 31 receives, via the antenna 23 and thecontroller communication module 19, operation data or the liketransmitted from the controller 7, and stores (temporarily) the data ina buffer area of the internal main memory 35 or the external main memory12.

The game apparatus body 5 is capable of transmitting/receiving imagedata, sound data and the like to/from the terminal device 6. The I/Oprocessor 31 outputs data of a game image (terminal game image)generated by the GPU 32 to the codec LSI 27. The codec LSI 27 performs apredetermined compression process on the image data supplied from theI/O processor 31. The terminal communication module 28 performs wirelesscommunication with the terminal device 6. Accordingly, the image datacompressed by the codec LSI 27 is transmitted by the terminalcommunication module 28 to the terminal device 6 via the antenna 29. Inthe exemplary embodiment, the codec LSI 27 compresses the image data byusing a highly efficient compression technique, for example, the H.264standard. The codec LSI 27 may adopt other compression techniques. Whenthe communication rate is sufficiently high, uncompressed image data maybe transmitted. The terminal communication module 28 is, for example, aWi-Fi certified communication module. The terminal communication module28 may perform wireless communication with the terminal device 6 at ahigh speed by using, for example, the technique of MIMO (Multiple InputMultiple Output) adopted in the IEEE802.11n standard, or may use othercommunication techniques.

The game apparatus body 5 transmits, to the terminal device 6, sounddata as well as the image data. That is, the I/O processor 31 outputssound data (terminal game sound) generated by the DSP 33 to the terminalcommunication module 28 via the codec LSI 27. The codec LSI 27 performsa compression process on the sound data in a similar manner to that forthe image data. Any compression technique may be adopted for the sounddata. In another embodiment, uncompressed sound data may be transmitted.The terminal communication module 28 transmits the compressed image dataand sound data to the terminal device 6 via the antenna 29.

The game apparatus body 5 transmits, in addition to the image data andsound data, various control data to the terminal device 6, according toneed. The control data represent control instructions for the componentsincluded in the terminal device 6, such as an instruction to controlon/off of a marker section (a marker section 65 shown in FIG. 5), and aninstruction to control image taking of a camera (a camera 66 shown inFIG. 5). The I/O processor 31 transmits the control data to the terminaldevice 6 in response to an instruction from the CPU 10.

The game apparatus body 5 can receive various data from the terminaldevice 6. Although details will be described later, in the exemplaryembodiment, the terminal device 6 transmits operation data, image data,and sound data. The respective data transmitted from the terminal device6 are received by the terminal communication module 28 via the antenna29. The image data and sound data transmitted from the terminal device 6have been subjected to a compression process similar to that for theimage data and sound data transmitted from the game apparatus body 5 tothe terminal device 6. Accordingly, these image data and sound data aretransmitted from the terminal communication module 28 to the codec LSI27, and subjected to a decompression process by the codec LSI 27. Thedecompressed data are outputted to the I/O processor 31. The operationdata, which has been received by the terminal communication module 28,is outputted to the I/O processor 31 via the codec LSI 27. The I/Oprocessor 31 stores (temporarily) the data received from the terminaldevice 6 in the buffer area of the internal main memory 35 or theexternal main memory 12.

The game apparatus body 5 is connectable to other devices and externalstorage media via the extension connector 20 and the memory cardconnector 21.

The game apparatus body 5 includes (on the front main surface thereof,for example) a power button 24, a reset button 25, an insertion slot inwhich the optical disc 4 is inserted, an eject button 26 for ejectingthe optical disc 4 from the insertion slot of the game apparatus body 5,and the like.

In another embodiment, some of the components of the game apparatus body5 may be constituted as an extension device separated from the gameapparatus body 5. In this case, the extension device may be connected tothe game apparatus body 5 via the extension connector 20. Specifically,the extension device may include, for example, the codec LSI 27, theterminal communication module 28, and the antenna 29, and may bedetachably connected to the extension connector 20. Thus, by connectingthe extension device to the game apparatus body which does not have theabove-mentioned components, the game apparatus body can be madecommunicable with the terminal device 6.

Next, a structure of the terminal device 6 will be described withreference to FIGS. 3 to 5. FIG. 3 is a diagram illustrating an exampleof an external structure of the terminal device 6. More specifically,(a) of FIG. 3 is a front view of the terminal device 6, (b) of FIG. 3 isa top view, (c) of FIG. 3 is a right side view, and (d) of FIG. 3 is abottom view. FIG. 4 shows an example of a state in which a user holdsthe terminal device 6 with both hands.

As shown in FIG. 3, the terminal device 6 includes a housing 60 whichgenerally has a horizontally long plate-like rectangular shape. Thehousing 60 is small enough to be held by the user.

The terminal device 6 includes the LCD 61 on a front surface of thehousing 60. The LCD 61 is provided near the center of the front surfaceof the housing 60. Therefore, as shown in FIG. 4, the user, holding thehousing 60 at portions to the right and left of the LCD 61, is allowedto move the terminal device 6 while viewing a screen of the LCD 61.

As shown in (a) of FIG. 3, the terminal device 6 includes, as operationmeans, a touch panel 62 on the screen of the LCD 61. In the exemplaryembodiment, the touch panel 62 is, but is not limited to, a resistivefilm type touch panel. A touch panel of any type, such as electrostaticcapacitance type, may be used. The touch panel 62 may be of single touchtype or multiple touch type. In the exemplary embodiment, the touchpanel 62 has the same resolution (detection accuracy) as that of the LCD61. However, the resolution of the touch panel 62 and the resolution ofthe LCD 61 may not necessarily be the same. Since the terminal device 6has the touch panel 62, the user is allowed to operate the touch panel62 while moving the terminal device 6. That is, the user is allowed todirectly (by using the touch panel 62) perform an input onto the screenof the LCD 61 while moving the LCD 61.

As shown in FIG. 3, the terminal device 6 has, as operation means, twoanalog sticks 63A and 63B, and a plurality of operation buttons 64A to64L. The analog sticks 63A and 63B are each a device for designating adirection. The analog sticks 63A and 63B are each configured such that astick part thereof to be operated by a finger of the user is slidable ortiltable in any direction (at any angle in any direction such as theupward, the downward, the rightward, the leftward, or the diagonaldirection) with respect to the front surface of the housing 60.

The operation buttons 64A to 64L are each assigned a function, asappropriate, in accordance with a game program. For example, the crossbutton 64A may be used for direction designation operation, selectionoperation, and the like, and the operation buttons 64E to 64H may beused for determination operation, cancellation operation, and the like.

As shown in (a) of FIG. 3, the terminal device 6 includes a markersection (the marker section 65 shown in FIG. 5) including a marker 65Aand a marker 65B, on the front surface of the housing 60. The markers65A and 65B are each constituted by one or more infrared LEDs. Themarker section 65 is used, like the marker 8, for causing the gameapparatus body 5 to calculate a movement or the like of the controller 7with respect to the marker section 65. The game apparatus body 5 iscapable of controlling the infrared LEDs of the marker section 65 to beon or off.

The terminal device 6 includes a camera 66. The camera 66 is provided onthe front surface of the housing 60. Accordingly, the camera 66 iscapable of taking an image of the face of the user holding the terminaldevice 6. For example, the camera 66 is capable of taking an image ofthe user playing a game while viewing the LCD 61.

The terminal device 6 has a microphone (a microphone 609 shown in FIG.5) as sound input means. The microphone 609 is embedded in the housing60 at a position inside a microphone hole 60 b. The microphone 609detects for a sound, such as the user's voice, around the terminaldevice 6.

The terminal device 6 has loudspeakers (loudspeakers 607 shown in FIG.5) as sound output means. A sound from the loudspeakers 607 is outputtedthrough speaker holes 60 a provided in the lower side surface of thehousing 60.

The terminal device 6 further includes an extension connector 67 forconnecting another device to the terminal device 6.

In the terminal device 6 shown in FIG. 3, the shapes of the operationbuttons and the housing 60, the number of each of the components, andthe positions at which the components are provided are merely examples.The shapes, numbers, and positions may be different from those describedabove.

Next, an internal structure of the terminal device 6 will be describedwith reference to FIG. 5. FIG. 5 is a block diagram illustrating anexample of an internal structure of the terminal device 6. As shown inFIG. 5, the terminal device 6 includes, in addition to the componentsshown in FIG. 3, a touch panel controller 601, a magnetic sensor 602, anacceleration sensor 603, the gyro sensor 604, a user interfacecontroller (UI controller) 605, a codec LSI 606, the loudspeakers 607, asound IC 608, the microphone 609, a wireless module 610, an antenna 611,an infrared communication module 612, a flash memory 613, a power supplyIC 614, a battery 615, and a vibrator 619. These electronic componentsare mounted on an electronic circuit board and accommodated in thehousing 60.

The UI controller 605 is a circuit for controlling data input to variousinput/output sections and data outputted from various input/outputsections. The UI controller 605 is connected to the touch panelcontroller 601, the analog stick 63 (the analog sticks 63A and 63B), theoperation button 64 (the operation buttons 64A to 64L), the markersection 65, the magnetic sensor 602, the acceleration sensor 603, thegyro sensor 604, and the vibrator 619. Further, the UI controller 605 isconnected to the codec LSI 606 and the extension connector 67. The powersupply IC 614 is connected to the UI controller 605, so that power issupplied to each of the components through the UI controller 605. Aninternal battery 615 is connected to the power supply IC 614, so thatpower is supplied from the battery 615. Further, a battery charger 616or a cable, which is supplied with power from an external power supply,may be connected to the power supply IC 614 via a connector or the like.In this case, the terminal device 6 can be supplied with power andcharged from the external power supply by using the battery charger 616or the cable.

The touch panel controller 601 is a circuit which is connected to thetouch panel 62 and controls the touch panel 62. The touch panelcontroller 601 generates a predetermined form of touch position data,based on a signal from the touch panel 62, and outputs the touchposition data to the UI controller 605. The touch position datarepresents coordinates of a position at which an input is performed onan input surface of the touch panel 62. The touch panel controller 601reads a signal from the touch panel 62 and generates touch position dataevery predetermined period of time. Further, various controlinstructions are outputted from the UI controller 605 to the touch panelcontroller 601.

The analog stick 63 outputs, to the UI controller 605, stick datarepresenting a direction in which the stick part slides (or tilts), andthe amount of the sliding (tilting). The operation button 64 outputs, tothe UI controller 605, operation button data representing an input stateof each of the operation buttons 64A to 64L (whether or not theoperation button has been pressed).

The magnetic sensor 602 detects the magnitude and direction of amagnetic field to detect an orientation. Orientation data representingthe detected orientation is outputted to the UI controller 605. The UIcontroller 605 outputs, to the magnetic sensor 602, a controlinstruction for the magnetic sensor 602. Examples of the magnetic sensor602 include: an MI (Magnetic Impedance) sensor, a fluxgate sensor, ahall sensor, a GMR (Giant Magneto Resistance) sensor, a TMR (TunnelingMagneto resistance) sensor, and an AMR (Anisotropic Magneto Resistance)sensor. However, any sensor may be adopted as long as the sensor candetect an orientation.

The acceleration sensor 603 is provided inside the housing 60. Theacceleration sensor 603 detects the magnitudes of linear accelerationsalong three axial directions (xyz axial directions shown in (a) of FIG.3). Acceleration data representing the detected accelerations isoutputted to the UI controller 605. The UI controller 605 outputs, tothe acceleration sensor 603, a control instruction for the accelerationsensor 603.

The gyro sensor 604 is provided inside the housing 60. The gyro sensor604 detects the angular velocities around the three axes (theabove-described xyz axes). Angular velocity data representing thedetected angular velocities is outputted to the UI controller 605. TheUI controller 605 outputs, to the gyro sensor 604, a control instructionfor the gyro sensor 604.

The vibrator 619 is, for example, a vibration motor or a solenoid. Theterminal device 6 is vibrated by actuating the vibrator 619 inaccordance with a control instruction outputted from the UI controller605 to the vibrator 619.

The UI controller 605 outputs, to the codec LSI 606, the operation data(hereinafter, referred to as terminal operation data) including thetouch position data, the stick data, the operation button data, theorientation data, the acceleration data, and the angular velocity data,which have been received from the respective components.

The codec LSI 606 is a circuit for performing a compression process ondata to be transmitted to the game apparatus body 5, and a decompressionprocess on data transmitted from the game apparatus body 5. The LCD 61,the camera 66, the sound IC 608, the wireless module 610, the flashmemory 613, and the infrared communication module 612 are connected tothe codec LSI 606. The codec LSI 606 further includes a CPU 617 and aninternal memory 618. Although the terminal device 6 is configured not toperform a game process, the terminal device 6 may execute a program formanaging the terminal device 6 or a program for communication. Forexample, a program stored in the flash memory 613 is loaded into theinternal memory 618 and executed by the CPU 617 when the terminal device6 is powered on, whereby the terminal device 6 is started up. A part ofthe area of the internal memory 618 is used as a VRAM for the LCD 61.

The camera 66 takes an image in accordance with an instruction from thegame apparatus body 5, and outputs data of the taken image to the codecLSI 606. The codec LSI 606 outputs, to the camera 66, a controlinstruction for the camera 66, such as an instruction to take an image.The camera 66 is also capable of taking a moving picture. That is, thecamera 66 is capable of repeatedly performing image taking, andrepeatedly outputting image data to the codec LSI 606.

The sound IC 608 is a circuit for controlling input of sound data fromthe microphone 609 and output of sound data to the loudspeakers 607.

The codec LSI 606 transmits the image data from the camera 66, the sounddata from the microphone 609, and the terminal operation data from theUI controller 605, to the game apparatus body 5 through the wirelessmodule 610. In the exemplary embodiment, the codec LSI 606 subjects theimage data and the sound data to a compression process similar to thatperformed by the codec LSI 27. The compressed image data and sound data,and the terminal operation data are outputted to the wireless module 610as transmission data. The antenna 611 is connected to the wirelessmodule 610, and the wireless module 610 transmits the transmission datato the game apparatus body 5 through the antenna 611. The wirelessmodule 610 has the same function as that of the terminal communicationmodule 28 of the game apparatus body 5. That is, the wireless module 610has a function of connecting to a wireless LAN by a method based on, forexample, the IEEE802.11n standard.

As described above, the transmission data transmitted from the terminaldevice 6 to the game apparatus body 5 includes the terminal operationdata, the image data, and the sound data. If another device is connectedto the terminal device 6 through the extension connector 67, datareceived from the other device may be included in the transmission data.The infrared communication module 612 performs, with another device,infrared communication based on, for example, the IRDA standard. Thecodec LSI 606 may transmit, to the game apparatus body 5, thetransmission data that includes data received by the infraredcommunication, according to need.

As described above, the compressed image data and sound data aretransmitted from the game apparatus body 5 to the terminal device 6.These data are received by the codec LSI 606 via the antenna 611 and thewireless module 610. The codec LSI 606 decompresses the received imagedata and sound data. The decompressed image data is outputted to the LCD61, and an image represented by the image data is displayed on the LCD61. On the other hand, the decompressed sound data is outputted to thesound IC 608, and a sound represented by the sound data is outputtedfrom the loudspeakers 607.

When control data is included in the data received from the gameapparatus body 5, the codec LSI 606 and the UI controller 605 makecontrol instructions for the respective components, according to thecontrol data. As described above, the control data represents controlinstructions for the respective components (in the exemplary embodiment,the camera 66, the touch panel controller 601, the marker section 65,the sensors 602 to 604, the vibrator 619, and the infrared communicationmodule 612) included in the terminal device 6. In the exemplaryembodiment, possible examples of the control instructions represented bythe control data are instructions to start and halt (stop) theoperations of the above-mentioned components. That is, some componentswhich are not used for a game may be halted to reduce power consumption.In this case, data from the halted components are not included in thetransmission data transmitted from the terminal device 6 to the gameapparatus body 5.

Next, an outline of information processing executed in the game system 1will be described with reference to FIG. 6 to FIG. 15.

The game apparatus body 5 in the game system 1 can communicate withanother game apparatus body via, for example, the Internet. In FIG. 6,for example, a game apparatus body 5 a can communicate with another gameapparatus body 5 b and a server device 9 which are connected to theInternet. The game apparatus body 5 a and the game apparatus body 5 bcan transmit and receive game data needed for a game to and from eachother. Accordingly, users (a user Ua1 and/or a user Ua2) of the gameapparatus body 5 a and users (a user Ub1 and/or a user Ub2) of the gameapparatus body 5 b can play a game together.

In the description below, “a” is appended to the reference charactersindicating components of a game system that includes the game apparatusbody 5 a shown in FIG. 6, while “b” is appended to the referencecharacters indicating components of a game system that includes the gameapparatus body 5 b shown in FIG. 6, according to need.

FIG. 7 illustrates a state of the game system that includes the gameapparatus body 5 a when the user (user Ua1) of the game apparatus body 5a and the user (user Ub1) of the game apparatus body 5 b are playing abowling game together.

In FIG. 7, a main game image is displayed on a monitor 2 a. A characterholding a bowling ball is displayed in the main game image. The user Ua1can operate the character by using a controller 7 a.

Meanwhile, a sub game image is displayed on an LCD 61 of a terminaldevice 6 a. The main game image is an image essential for playing of thegame, whereas the sub game image is not essential for playing of thegame. The sub game image is an image for providing various informationabout the game to the users. In the example of FIG. 7, scores of thebowling game are displayed on the LCD 61 of the terminal device 6 a.

As shown in FIG. 8, the sub game image displayed on the LCD 61 includesa video chat start button B1 for making an instruction to start videochat. The user Ua2 can have video chat with the user (user Ub2) of thegame apparatus body 5 b by touching the video chat start button B1.

When the user Ua2 touches the video chat start button B1, the face ofthe user Ub2 is displayed in real time on the LCD 61 of the terminaldevice 6 a, as shown in FIG. 9. In addition, the voice of the user Ub2is outputted in real time from loudspeakers 607 of the terminal device 6a. Meanwhile, image data (camera image) inputted from a camera 66 of theterminal device 6 a, and sound data (microphone sound) inputted from amicrophone 609 of the terminal device 6 a are transmitted to the gameapparatus body 5 b through the game apparatus body 5 a. Accordingly, theface of the user Ua2 is displayed in real time on an LCD 61 of aterminal device 6 b in the game system that includes the game apparatusbody 5 b, and the voice of the user Ua2 is outputted in real time fromloudspeakers 607 of the terminal device 6 b.

Thus, as shown in FIG. 10, when the user Ua1 and the user Ub1 areplaying the game, the user Ua2 and the user Ub2 are allowed to enjoy aconversation in real time while viewing each other's face.

When the user Ua2 and the user Ub2 are having video chat, a game imageis displayed on the monitor 2 a in the same way as usual, and gamesounds are outputted from loudspeakers 2Ra and 2La in the same way asusual. Therefore, it does not become difficult for the user Ua1 to viewthe game image and catch the game sounds.

In FIG. 9, when the user Ua2 touches an end button B2, the video chat isended, and then the game image (sub game image) as shown in FIG. 8 isdisplayed again on the LCD 61 of the terminal device 6 a.

The game system 1 can be used for applications other than games. Forexample, as shown in FIG. 11, a user can view television programs bymeans of the monitor 2. In this case, a monitor control image as shownin FIG. 12 is displayed on the LCD 61 of the terminal device 6. When theuser touches a desired one of a plurality of channel selection buttonsB3 displayed in the monitor control image, an infrared signal (monitorcontrol signal) corresponding to the touched channel selection button B3is outputted from the infrared communication module 612 of the terminaldevice 6 (see FIG. 11). Thus, the user can switch broadcast channels(television programs) to be displayed on the monitor 2 by operating theterminal device 6.

In the monitor control image, a channel selection button B3corresponding to a broadcast channel that the user is viewing may behighlighted. In the example of FIG. 12, the channel selection button B3corresponding to the 6th channel is highlighted. The channel selectionbutton B3 corresponding to the 6th channel remains highlighted until theuser touches another channel selection button B3.

As shown in FIG. 12, a chat start button B4 is also displayed in themonitor control image. If the user touches the chat start button B4,he/she can enjoy chat with another user who is viewing the samebroadcast channel (television program).

In FIG. 12, when the user touches the chat start button B4, useridentification information indicating the user, and currently-viewedchannel information indicating a broadcast channel that the user iscurrently viewing (6th channel in the example of FIG. 12), are uploadedfrom the game apparatus body 5 into the server device 9 (see FIG. 6).Further, user identification information (user names, for example)indicating other users who are viewing the same television program,connection information (global IP addresses of the other users, forexample), and the like, are downloaded from the server device 9 into thegame apparatus body 5. In this manner, preparations for chat between theusers are completed. Alternatively, a friend list D3 or the like, whichwill be described later, may be uploaded from the game apparatus body 5into the server device 9, whereby user identification information,connection information, and the like about preregistered users (friends,for example) among the other users who are viewing the same televisionprogram may be downloaded from the server device 9 into the gameapparatus body 5.

When preparations for chat between the users have been completed, a chatimage as shown in FIG. 13 is displayed on the LCD 61 of the terminaldevice 6. Characters and user names corresponding to the other usersviewing the same television program are displayed in the chat image.Referring to the example of FIG. 13, it is understood that two users,“Ichiro” and “Kei”, are viewing the television program of the 6thchannel.

In FIG. 13, when the user touches a user selection button B5 associatedwith another user to whom the user wants to send a message, a massageinput image as shown in FIG. 14 is displayed. Thereafter, when the userinputs a message by using, for example, a software key board and touchesa transmission button B7, the message is transmitted to the other userselected by means of the user selection button B5.

When a message arrives from the other user, the received message isdisplayed in the chat image, as shown in FIG. 15.

In FIG. 13 or FIG. 15, when the user touches an end button B6, the chatis ended, and the monitor control image as shown in FIG. 12 is displayedagain on the LCD 61 of the terminal device 6.

Hereinafter, operations of the game system 1 for realizing theinformation processing will be described in detail with reference toFIG. 16 to FIG. 19.

FIG. 16 illustrates an example of various data stored in the externalmain memory 12 of the game apparatus body 5 when the informationprocessing is executed.

An information processing program D1 is a program for causing the CPU 10of the game apparatus body 5 to execute the information processing. Forexample, the information processing program D1 is loaded into theexternal main memory 12 from the optical disc 4.

User identification information D2 is information (user name, forexample) for identifying a user of the game apparatus body 5.

The friend list D3 is information for identifying the user's friends.The user can select a desired friend among his/her friends registered inthe friend list D3, and can play a network game together with theselected friend.

Controller operation data D4 is operation data which is periodicallytransmitted from the controller 7.

Terminal operation data D5 is operation data which is periodicallytransmitted from the terminal device 6. As described above, the terminaloperation data D5 includes the touch position data and the like.

Transmission data buffer D6 is a storage area for temporarily storingnetwork game data D61 and chat data D62 which are to be transmitted toanother game apparatus body 5. Network game data is data transmitted andreceived to and from another game apparatus body 5 in order for thenetwork game to be executed. The network game data may be the controlleroperation data, for example. The chat data includes image data, sounddata, and character data which are inputted via the camera 66, themicrophone 609, and the touch panel 62, respectively, of the terminaldevice 6.

Reception data buffer D7 is a storage area for temporarily storingnetwork game data D71 and chat data D72 which are received from anothergame apparatus body 5.

The currently-viewed channel information is information indicating abroadcast channel that the user is currently viewing.

Next, the flow of the information processing, which is executed by theCPU 10 of the game apparatus body 5 based on the information processingprogram D1, will be described with reference to flowcharts shown in FIG.17 to FIG. 19.

When the execution of the information processing program D1 is started,the CPU 10 determines, at step S10 in FIG. 17, whether or not to executethe network game. For example, the CPU 10 makes the determination bydisplaying a menu image on the monitor 2 or on the LCD 61 of theterminal device 6, and then causing the user to select whether thenetwork game is executed or monitor control is performed. If the networkgame is executed, the processing proceeds to step S11. Otherwise, theprocessing proceeds to step S12.

At step S11, the CPU 10 executes a network game process. The networkgame process will be described in detail later. When the network gameprocess ends, the CPU 10 ends the execution of the informationprocessing program D1.

At step S12, the CPU 10 determines whether or not to perform monitorcontrol. If the monitor control is performed, the processing proceeds tostep S13. Otherwise, the processing returns to step S10.

At step S13, the CPU 10 executes a monitor control process. The monitorcontrol process will be described later. When the monitor controlprocess ends, the CPU 10 ends the execution of the informationprocessing program D1.

Next, the network game process at step S11 in FIG. 17 will be describedin detail with reference to FIG. 18.

At step S20, the CPU 10 selects another user who is to be acommunication partner playing the network game. Various methods forselecting a communication partner are conceivable. For example, the usermay be allowed to select one or more desired friends among his/herfriends registered in the friend list D3. Alternatively, a method may beused in which the game apparatus body 5 makes an inquiry to the serverdevice 9, and the server device 9 introduces other users who can playthe network game.

At step S21, the CPU 10 connects to the game apparatus body 5 of thecommunication partner selected at step S20. For example, in the casewhere the communication partner is selected among the user's friendsregistered in the friend list D3, the CPU 10 may transmit identificationinformation of the selected friend to the server device 9, obtain aglobal IP address of the game apparatus body 5 of the friend, andtransmit a connection request to the game apparatus body 5 of thefriend.

At step S22, the CPU 10 obtains the controller operation data D4 and theterminal operation data D5 from the controller 7 and the terminal device6, respectively.

At step S23, the CPU 10 determines whether or not video chat is beingconducted. If video chat is being conducted, (in a period of time fromwhen the user touches the video chat start button B1 in FIG. 8 to whenthe user touches the end button B2 in FIG. 9, for example), theprocessing proceeds to step S26. Otherwise, the processing proceeds tostep S24.

At step S24, the CPU 10 transmits the network game data D61 to the gameapparatus body 5 of the communication partner.

At step S25, the CPU 10 receives the network game data 71 from the gameapparatus body 5 of the communication partner.

At step S26, the CPU 10 transmits the network game data D61 and the chatdata D62 to the game apparatus body 5 of the communication partner. Atthis time, since video chat is being conducted, the chat data D62 to betransmitted includes image data and sound data which are inputted fromthe camera 66 and the microphone 609, respectively, of the terminaldevice 6.

At step S27, the CPU 10 receives the network game data 71 and the chatdata D72 from the game apparatus body 5 of the communication partner.

At step S28, the CPU 10 executes a game process (control of charactersin the game world, for example), based on the controller operation dataD4 and the received network game data D71.

At step S29, the CPU 10 generates a monitor game image, and outputs themonitor game image to the monitor 2. The outputted monitor game image isdisplayed on the monitor 2. The GPU 32 may execute part or the whole ofthe process of generating the monitor game image in accordance with aninstruction from the CPU 10.

At step S30, the CPU 10 determines whether or not video chat is beingconducted. If video chat is being conducted, the processing proceeds tostep S33. Otherwise, the processing proceeds to step S31.

At step S31, the CPU 10 generates a terminal game image (the sub gameimage in FIG. 8, for example), and outputs the terminal game image tothe terminal device 6. The outputted terminal game image is displayed onthe LCD 61 of the terminal device 6. The GPU 32 may execute part or thewhole of the process of generating the terminal game image in accordancewith an instruction from the CPU 10.

At step S32, the CPU 10 outputs game sounds to the loudspeakers 2R and2L.

At step S33, the CPU 10 outputs, to the terminal device 6, the receivedchat data (image data and sound data). Thus, the face of thecommunication partner is displayed on the LCD 61 of the terminal device6, and the voice of the communication partner is outputted from theloudspeakers 607 of the terminal device 6.

At step S34, the CPU 10 determines whether or not the game has beenended. If the game has been ended, the network game process is ended.Otherwise, the processing returns to step S22.

Next, the monitor control process at step S13 in FIG. 17 will bedescribed in detail with reference to FIG. 19.

At step S40, the CPU 10 generates a monitor control image (the monitorcontrol image in FIG. 12, for example), and outputs the monitor controlimage to the terminal device 6. The outputted monitor control image isdisplayed on the LCD 61 of the terminal device 6.

At step S41, the CPU 10 obtains the terminal operation data D5 from theterminal device 6.

At step S42, the CPU 10 determines, based on the terminal operation dataD5, whether or not a channel selection instruction has been inputted(whether or not any one of the channel selection buttons B3 in FIG. 12has been touched, for example). If a channel selection instruction hasbeen inputted, the processing proceeds to step S43. Otherwise, theprocessing proceeds to step S45.

At step S43, the CPU 10 causes the terminal device 6 to output a monitorcontrol signal, in accordance with a broadcast channel selected throughthe channel selection instruction. Specifically, the CPU 10 detects theposition, on the monitor control image, that has been touched by theuser, based on the touch position data included in the terminaloperation data D5. Then, the CPU 10 transmits control data to theterminal device 6 such that a monitor control signal corresponding tothe position touched by the user is outputted from the infraredcommunication module 612 of the terminal device 6.

At step S44, the CPU 10 updates the currently-viewed channel informationD8, in accordance with the broadcast channel selected through thechannel selection instruction.

At step S45, the CPU 10 determines whether or not a chat startinstruction has been inputted (whether or not the chat start button B4in FIG. 12 has been touched, for example). If the chat start instructionhas been inputted, the processing proceeds to step S46. Otherwise, theprocessing proceeds to step S51.

At step S46, the CPU 10 uploads the currently-viewed channel informationD8, the user identification information D2, and the like, into theserver device 9.

The server device 9 retains currently-viewed channel information D8 anduser identification information D2 uploaded from a plurality of gameapparatus bodies 5. Accordingly, when currently-viewed channelinformation 8 and user identification information D2 are uploaded intothe server device 9 from still another game apparatus body 5, the serverdevice 9 can provide, to the still other game apparatus body 5, useridentification information D2 of users who are viewing the sametelevision program that the user of the still other game apparatus body5 is viewing.

At step S47, the CPU 10 downloads, from the server device 9, useridentification information D2 of other users who are viewing the sametelevision program that the user of the game apparatus body 5 includingthe CPU 10 is viewing.

At step S48, the CPU 10 executes a chat process, based on the terminaloperation data D5. Specifically, the CPU 10 executes, for example, thefollowing processes: a process of transmitting, to the game apparatusbody 5 of another user specified by the user, the chat data D62representing a message (character data) inputted by the user (see FIG.14); and a process of generating a chat image that includes a message(character data) included in the chat data D72 received from anothergame apparatus body 5, and displaying the chat image on the LCD 61 (seeFIG. 15).

At step S49, the CPU 10 determines whether or not a chat end instructionhas been inputted (whether or not the end button B6 in FIG. 13 or FIG.15 has been touched, for example). If a chat end instruction has beeninputted, the processing proceeds to step S50. Otherwise, the processingreturns to step S47.

At step S50, the CPU 10 transmits a chat end notification to the serverdevice 9. Then, the currently-viewed channel information D8 and the useridentification information D2 of the user of the game apparatus body 5is deleted from the server device 9.

At step S51, the CPU 10 determines whether the monitor control has beenended. If the monitor control has been ended, the monitor controlprocess ends. Otherwise, the processing returns to step S40.

As described above, according to the exemplary embodiment, when a user(main user) is playing a game by means of the monitor 2 or when he/sheis viewing a television program by means of the monitor 2, another user(sub user) is allowed to have video chat or text chat by using the LCD61 of the terminal device 6. Therefore, it is possible to have chatwithout affecting playing of a game or viewing of a television program.

Further, according to the exemplary embodiment, a game process and achat process can be executed independently in parallel by means of asingle game apparatus 3. Therefore, when a user (user A) is playing agame, another user (user B) can have chat by using the terminal device 6of the game apparatus 3. By having chat using the terminal device 6, auser can feel as if he/she is joining a game without actually joiningthe game, and thus even a person who is poor at games can lightheartedlyget enjoyment. In addition, since the terminal device 6 includes thetouch panel 62, even a person who is unfamiliar with games can easilyperform a menu operation and a character input operation. Further, if auser who is not interested in games has chat when another user isplaying a game, this may lead the user to find games to be fun and haveinterest in games.

Further, according to the exemplary embodiment, when a main user isplaying a network game, a sub user can start video chat by just touchingthe video chat start button B1 (FIG. 8). This saves the sub user thetrouble of selecting a chat partner or inputting address information ofa chat partner in order to start video chat. Further, by joining chat,even a user who is poor at playing games can spend enjoyable timetogether with another user who is playing a network game.

Further, according to the exemplary embodiment, game sounds areoutputted from the loudspeakers 2R and 2L, and the voice of a chatpartner is outputted from the loudspeakers 607 of the terminal device 6.Therefore, game sounds do not become difficult for a main user to catch,and the voice of a chat partner does not become difficult for a sub userto distinguish.

Further, according to the exemplary embodiment, broadcast channels(television programs) to be displayed on the monitor 2 can be switchedby means of the terminal device 6. In addition, a user can easily start,in accordance with his/her desire, chat with another user who is viewingthe same television program, and can enjoy chatting about their commontopic, i.e., the television program they are viewing.

The above-described embodiment is only an example.

In the above-described embodiment, a network game is played, or chat isconducted, between two game apparatuses 3. Alternatively, in anotherembodiment, a network game may be played, or chat may be conducted,among three or more game apparatuses 3. In the case where video chat isconducted among three or more game apparatuses 3, the screen of the LCD61 may be divided, and a plurality of users who are communicationpartners may be displayed at the same time.

Further, in the above-described embodiment, when a network game is beingplayed by means of the monitor 2, video chat is conducted by means ofthe terminal device 6. Alternatively, text chat may be conducted insteadof video chat.

Further, in the above-described embodiment, when a television program isbeing viewed by means of the monitor 2, text chat is conducted by meansof the terminal device 6. Alternatively, video chat may be conductedinstead of text chat.

Further, in the above-described embodiment, when a user is viewing atelevision program by means of the monitor 2, user identificationinformation and the like of other users who are viewing the sametelevision program are obtained from the server device 9 to allow theuser to chat with another user. Alternatively, in another embodiment, auser may be allowed to select a desired friend among his/her friendsregistered in the friend list D3, and to chat with the selected friend.

Further, in the above-described embodiment, game sounds are outputtedfrom the loudspeakers 2R and 2L, and the voice of a chat partner isoutputted from the loudspeakers 607 of the terminal device 6.Alternatively, in another embodiment, game sounds may be outputted fromthe loudspeakers 607 of the terminal device 6. Similarly, the voice of achat partner may be outputted from the loudspeakers 2R and 2L.

Further, in the above-described embodiment, a single computer (CPU 10)executes the plurality of processes shown in FIG. 17 to FIG. 19.Alternatively, in another embodiment, a plurality of computers may sharethe execution of the plurality of processes. In still anotherembodiment, part of the plurality of processing may be realized by ahardware circuit.

Further, in the above-described embodiment, the game apparatus 3executes the image processing. Alternatively, in another embodiment, aninformation processing apparatus (general-purpose computer, for example)other than the game apparatus may execute the image processing.

Further, in the above-described embodiment, a single informationprocessing apparatus (game apparatus body 5) executes the plurality ofprocesses shown in FIG. 17 to FIG. 19. Alternatively, in anotherembodiment, a plurality of information processing apparatuses (gameapparatus body 5 and terminal device 6, for example) may share theexecution of the plurality of processes.

Further, in the above-described embodiment, the information processingprogram D1 is supplied to the game apparatus body 5 from the opticaldisc 4. Alternatively, in another embodiment, the information processingprogram D1 may be supplied to the game apparatus body 5 from anycomputer-readable storage medium (such as a CD-ROM, a semiconductormemory, or the like). In still another embodiment, the informationprocessing program D1 may be previously stored in a non-volatile memory(ROM/RTC 13, flash memory 17) inside the game apparatus body 5 In stillanother embodiment, the information processing program D1 may betransmitted to the game apparatus body 5 from another informationprocessing apparatus (game apparatus or server device).

The systems, devices and apparatuses described herein may include one ormore processors, which may be located in one place or distributed in avariety of places communicating via one or more networks. Suchprocessor(s) can, for example, use conventional 3D graphicstransformations, virtual camera and other techniques to provideappropriate images for display. By way of example and withoutlimitation, the processors can be any of: a processor that is part of oris a separate component co-located with the stationary display and whichcommunicates remotely (e.g., wirelessly) with the movable display; or aprocessor that is part of or is a separate component co-located with themovable display and communicates remotely (e.g., wirelessly) with thestationary display or associated equipment; or a distributed processingarrangement some of which is contained within the movable displayhousing and some of which is co-located with the stationary display, thedistributed portions communicating together via a connection such as awireless or wired network; or a processor(s) located remotely (e.g., inthe cloud) from both the stationary and movable displays andcommunicating with each of them via one or more network connections; orany combination or variation of the above.

The processors can be implemented using one or more general-purposeprocessors, one or more specialized graphics processors, or combinationsof these. These may be supplemented by specifically-designed ASICs(application specific integrated circuits) and/or logic circuitry. Inthe case of a distributed processor architecture or arrangement,appropriate data exchange and transmission protocols are used to providelow latency and maintain interactivity, as will be understood by thoseskilled in the art.

Similarly, program instructions, data and other information forimplementing the systems and methods described herein may be stored inone or more on-board and/or removable memory devices. Multiple memorydevices may be part of the same device or different devices, which areco-located or remotely located with respect to each other.

While the exemplary embodiments have been described in detail, theforegoing description is only illustrative in all aspects, and is notintended to limit the scope of the appended claims. It is understoodthat numerous other modifications and variations can be devised.

1. A computer-readable storage medium having stored therein aninformation processing program executed in an information processingapparatus that is capable of displaying images on a stationary displaydevice and on a portable display device having a predetermined inputsection, and that includes an input device and a communicator configuredto communicate with another information processing apparatus that is acommunication counterpart apparatus, the information processing programcausing a computer of the information processing apparatus to functionas: a processor configured to execute a predetermined informationprocess, based on an input from the input device; a first displaycontroller configured to cause an image which is based on thepredetermined information process to be displayed on the stationarydisplay device; a transmitter configured to transmit, to thecommunication counterpart apparatus through the communicator, characterinformation or a camera image inputted via the predetermined inputsection; a receiver configured to receive, from the communicationcounterpart apparatus through the communicator, character information ora camera image inputted on the communication counterpart apparatus; anda second display controller configured to cause the characterinformation or the camera image that is received by the receiver to bedisplayed on the portable display device.
 2. The computer-readablestorage medium according to claim 1, wherein the predeterminedinformation process is a game process.
 3. The computer-readable storagemedium according to claim 2, wherein the transmitter transmits game datafor a network game process to the communication counterpart apparatusthrough the communicator, in addition to the character information orthe camera image that is inputted via the predetermined input section,the receiver receives game data for the network game process from thecommunication counterpart apparatus through the communicator, inaddition to the character information or the camera image that isinputted on the communication counterpart apparatus, and the processorexecutes the network game process, based on the input from the inputdevice and the game data that is received from the communicationcounterpart apparatus.
 4. The computer-readable storage medium accordingto claim 2, wherein the information processing apparatus is connected toa first loudspeaker, the predetermined input section includes a cameraand a microphone, the portable display device includes a secondloudspeaker, the transmitter transmits, to the communication counterpartapparatus through the communicator, a camera image inputted via thecamera and a microphone sound inputted via the microphone, the receiverreceives, from the communication counterpart apparatus through thecommunicator, a camera image and a microphone sound that are inputted onthe communication counterpart apparatus, and the information processingprogram further causes the computer to function as: a first soundcontroller configured to generate a game sound in accordance with thegame process, and to cause the game sound to be outputted from the firstloudspeaker; and a second sound controller configured to cause themicrophone sound that is received by the receiver to be outputted fromthe second loudspeaker.
 5. The computer-readable storage mediumaccording to claim 1, wherein the second display controller includes: animage generator configured to generate an image to be displayed on theportable display device, based on the character information or thecamera image that is received by the receiver; and an image outputterconfigured to output the image generated by the image generator to theportable display device by wireless transmission.
 6. Thecomputer-readable storage medium according to claim 1, wherein thesecond display controller includes: an image generator configured togenerate an image to be displayed on the portable display device, basedon the character information or the camera image that is received by thereceiver; and an image outputter configured to compress the imagegenerated by the image generator, and to output the compressed image tothe portable display device.
 7. The computer-readable storage mediumaccording to claim 1, wherein the predetermined input section includes atouch panel, and the character information is inputted by means of thetouch panel.
 8. The computer-readable storage medium according to claim1, wherein the stationary display device has a function of receiving anddisplaying television broadcasts, and the portable display device has afunction of outputting a control signal to the stationary display devicein accordance with a channel selection instruction inputted through thepredetermined input section, and thus switching television channels tobe displayed on the stationary display device.
 9. The computer-readablestorage medium according to claim 8, wherein the information processingapparatus is communicable with a server device through the communicator,and the information processing program further causes the computer tofunction as: an uploader configured to upload, into the server device,channel information indicating a television channel that a user of theinformation processing apparatus is viewing, and user identificationinformation for identifying the user, in accordance with the channelselection instruction; a downloader configured to download, from theserver device, user identification information that is associated withthe channel information uploaded by the uploader among useridentification information uploaded from other information processingapparatuses into the server device; and a communication counterpartdeterminer configured to determine the communication counterpartapparatus, based on the user identification information downloaded bythe downloader.
 10. The computer-readable storage medium according toclaim 1, wherein the information processing apparatus further includes astorage section configured to store user identification informationabout users of other information processing apparatuses, and theinformation processing program further causes the computer to functionas a communication counterpart determiner configured to determine thecommunication counterpart apparatus, based on the user identificationinformation stored in the storage section about the users of the otherinformation processing apparatuses.
 11. The computer-readable storagemedium according to claim 10, wherein the communication counterpartdeterminer selects user identification information among the useridentification information stored in the storage section about the usersof the other information processing apparatuses, based on an input fromthe input device or the predetermined input section, and determines, asthe communication counterpart apparatus, another information apparatusthat is associated with the selected user identification information.12. The computer-readable storage medium according to claim 1, whereinthe predetermined input section includes a microphone, the portabledisplay device includes a loudspeaker, the transmitter transmits amicrophone sound inputted via the microphone, to the communicationcounterpart apparatus through the communicator, in addition to thecharacter information or the camera image, the receiver receives amicrophone sound inputted on the communication counterpart apparatus,from the communication counterpart apparatus through the communicator,in addition to the character information or the camera image that isinputted on the communication counterpart apparatus, and the imageprocessing program further causes the computer to function as a soundcontroller configured to cause the microphone sound that is received bythe receiver to be outputted from the loudspeaker.
 13. An informationprocessing apparatus capable of displaying images on a stationarydisplay device and on a portable display device having a predeterminedinput section, the information processing apparatus comprising: an inputdevice; a communicator configured to communicate with anotherinformation processing apparatus that is a communication counterpartapparatus; a processor configured to execute a predetermined informationprocess, based on an input from the input device; a first displaycontroller configured to cause an image which is based on thepredetermined information process to be displayed on the stationarydisplay device; a transmitter configured to transmit, to thecommunication counterpart apparatus through the communicator, characterinformation or a camera image inputted via the predetermined inputsection; a receiver configured to receive, from the communicationcounterpart apparatus through the communicator, character information ora camera image inputted on the communication counterpart apparatus; anda second display controller configured to cause the characterinformation or the camera image by the receiver to be displayed on theportable display device.
 14. An information processing system capable ofdisplaying images on a stationary display device and on a portabledisplay device having a predetermined input section, the informationprocessing system comprising: an input device; a communicator configuredto communicate with another information processing system that is acommunication counterpart system; a processor configured to execute apredetermined information process, based on an input from the inputdevice; a first display controller configured to cause an image which isbased on the predetermined information process to be displayed on thestationary display device; a transmitter configured to transmit, to thecommunication counterpart system through the communicator, characterinformation or a camera image inputted via the predetermined inputsection; a receiver configured to receive, from the communicationcounterpart system through the communicator, character information or acamera image inputted on the communication counterpart system; and asecond display controller configured to cause the character informationor the camera image that is received by the receiver to be displayed onthe portable display device.
 15. An information processing methodperformed by an information processing apparatus that is capable ofdisplaying images on a stationary display device and on a portabledisplay device having a predetermined input section and that includes aninput device and a communicator configured to communicate with anotherinformation processing apparatus that is a communication counterpartapparatus, the information processing method comprising the steps of:executing a predetermined information process, based on an input fromthe input device; causing an image which is based on the predeterminedinformation process to be displayed on the stationary display device;transmitting, to the communication counterpart apparatus through thecommunicator, character information or a camera image inputted via thepredetermined input section; receiving, from the communicationcounterpart apparatus through the communicator, character information ora camera image inputted on the communication counterpart apparatus; andcausing the character information or the camera image that is receivedfrom the communication counterpart apparatus to be displayed on theportable display device.